Fast Simultaneous Tracking and Recognition Using Incremental Keypoint Matching

This paper presents a unified approach to object recognition and object tracking, combining local feature matching with optical flow. Like many traditional recognition algorithms, the one described here implements recognition by matching detected image patches against a database of known objects. This algorithm, however, matches keypoints incrementally, meaning that it only tests a few keypoints at each frame until the complete object is identified. Recognition and tracking thus proceed in real-time, even with high dimensional features and an arbitrarily large database. Central to this work is the system by which keypoint matching and optical flow mutually aid one another. Keypoint matching recognizes an object and estimates its pose in order to initialize tracking. Optical flow tracking, in turn, maintains the object pose over subsequent frames, discarding newly matched keypoints that do not fit with the current pose estimation. Experimental results demonstrate that this powerful combination provides robust, real-time recognition and tracking of multiple objects in the presence of scale and orientation changes as well as partial occlusion.